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ABSTRACT 


Let  f be  a continuous  density  function  which  has  coa^act  support  [a,  b] . 

Let  W be  a nonnegative  weight  function  which  is  continuous  on  its  coiopact 
b 

support  [a,  b]  and  /w(x)dx  = 1.  The  cot?)lete  convergence  of 
a 


sup 


1 

nbtn) 


I w 


k=l 


. b(n)  , 


- f(s) 


to  zero  is  obtained  under  varying  conditions  on  the  bandwidths  b(n)  and 

smoothness  of  W.  For  example,  one  choice  of  the  weight  function  is 

2 6 

Epanechnikov ' s optimal  function  and  nb  (n)  > n for  some  6 > 0.  The  uniform 
complete  convergence  of  the  mode  estimate  is  also  considered. 


Key  Words  and  Phrases.  Density  function  estimates,  weight  function,  bandwidth, 
complete  convergence,  and  subGuassian  random  variables. 
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1.  Introduction  and  Preliminaries.  The  construction  of  a family  of 

■■  I 'r,  ' I . - . ,■  ',1. 

estimates  of  a density  function  f(x]  and  of  the  mode  has  been  studied  by 
several  people.  Rosenblatt  (1956)  considered  a general  class  of  density 
estimates : 


fn(x) 


(ic  - X. 


b(n) 


(1.1) 


where  X^  are  i.i.d.  random  variables  with  continuous  density  function 

f(x),  W(x)  is  a bounded  integrable  weight  function  such  that 


/ W(x)dx  - 1 


?.nd  b(n)  is  a bandwidth  that  tends  to  zero  as  n -►  *.  Thus,  the  question 
arises  as  to  suitable  choices  of  W(x)  and  b(n)  so  that  the  estimate  function 
fj^(x)  is  optimal  (in  some  sense).  The  local  properties  of  the  ostlmate- function 
in  (1.1)  have  been  studied  extensively  [see  Rosenblatt  (1971)  for  a general 
survey],  and  a global  measure  of  deviation  of  the  curve  f^(x)  from  f(x)  by 

Ilf  - f||„  = sup  If  (X)  - f(x)|  (1.2) 

xeR 

has  been  considered.  Parzen  (1962)  showed  that  if  the  (true)  underlying 
density  function  f(x)  is  uniformly  continuous  then  | |fj^  - f | |^  converges  in 
probability  to  zero  under  the  following  conditions: 

(PI)  - / e^^'Sf(x)dx  is  absolutely  integrable, 

•OD 

(P2)  »'nb(n)  • as  n -►  «. 

The  results  of  Nadaraya  (1965)  and  V/oodroofe  (1967)  on  uniform  consistency  in 
the  strong  sense  are  listed  in  the  last  section  for  comparison  with  the  results 
of  this  paper. 
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The  major  results  of  this  paper  give  a new  class  of  "good"  weight 
functions  [which  includes  the  optimal  function]  under  mild  conditions  on 
the  bandwidth  sequence  b(n)  where  uniform  consistency  of  the  estimate 
is  obtained  by  the  conplete  convergence  [see  Stout  (1966)]  of  ||f^  - f 1 1^ 
to  zero.  The  main  tools  used  in  obtaining  these  results  will  be  the  /smoothness 
of  the  weight  function  and  sub-Gaussian  techniques. 

Throughout  this  paper,  attention  will  be  restricted  to  a density  function 

which  has  compact  support  [a,  b]  and  weight  functions  W(x)  which  satisfy 
b 

(i)  / N(x)dx  » 1 and 

a 

(ii)  W(x}  is  nonnegative  and  continuous  on  [a,  b]  and  vanish  outside  [a,  b]. 

Let  be  a polygonal  approximating  function  on  the  space  of  continuous  functions 
with  domain  [a,  b],  C[a,  b] . That  is, 

«[•  * ^ ii  - 

for  i ■ 0,  1,  ...,  n and  g e C[a,  b] , and  is  linear  between  the  points 

a ♦ and  a ♦ Recall  that  the  modulus  of  continuity, 

n n 

“j(«) , is  defined  by  Billingsley  (1968)  by 


“ (y)  ■ sup  |g(t)  - g(s)| 

« It-slsy 


(1.3) 


for  Y > 0«  s,  t e [a,  b],  and  g e C[a,  b]. 

Definition  [Chow  (1966)1  A random  variable  X is  said  to  be  sub-Gaussian 
if  there  exists  a 2 0 such  that 


E[exp(tX)]  i exp 


for  all  t e R. 


(1.4) 
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If  X is  sub-Gaussian,  then  let 

t(X)  a inf{a  i 0:  Inequality  (1.4)  holds}. 

Some  basic  properties  on  sub-Gaussian  random  variables  include: 

(1)  If  P[|X|  s K]  ■ 1 and  EX  = 0,  then  E[exp(tX)]  s exp(K^t^).  (1.5) 

(2)  If  t(X)  = a,  then  P[|xl  s X]  s 2 exp(-X^/2o^) . (1.6) 

(3)  The  sum  of  two  independent  sub-Gaussian  random  variables  is  sub-Gaussian. 
Finally,  a sequence  of  random  variables  {X^}  is  said  to  converge  completely 

to  a random  variable  X if 

Cl^^l^n  - X|  > e]  < - (1.7) 

for  each  e > 0.  Thus,  coiiq>lete  convergence  implies  convergence  with  probability 
one  by  Borel's  inequality. 
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2.  Main  Results.  In  this  section  the  complete  convergence  of  | jf^^  - f 1 
to  zero  is  obtained  under  conditions  on  the  modulus  of  continuity  of  the 
weight  function  W(x)  and  the  rate  of  convergence  to  zero  by  the  bandwidth  b(n). 
Also,  the  uniform  consistency  of  the  mode  estimate  is  obtained  in  this  setting. 
The  uniform  consistency  of  the  estimate  [in  the  con^lcte  sense]  is 

accomplished  by  two  lemmas. 


2 6 

Lemma  1.  If  (i)  nb  (n)  > n for  some  6 > 0 and  (ii)  w. 


for  some  integer  r > 0,  then 


2b  - 2a 


n^b(n)  J 


o(b(n)) 


sup 

-eB<5<oa 


f (S)  - EW 


S - X, 


(2.1) 


completely  as  n • where  fjj(x)  is  defined  in  (1.1). 


Proof;  Since  W and  fy  vanish  outside  [a,  b]  and  b(n)  -►  0,  the  sup  in 


a,  2b]  suffices  if  a > 0).  Let 

X,  ; 

for  each  k » 1 , . . . , n . 


EW 


s - 


'1 


b(n)J 


(2.1)  need  only  be  taken  over  [2a,  2b]  ( 

j « ~ and  let  W.(s)  = W s - 

" n%(nl  I raj; 

Thus,  EWj^(s)  » 0 for  each  s c [a,  b]  and  each  k.  Furthermore, 

|\(s)  - \(t)l 


t-s 


i sup 
It-sUd. 


^k 

L "xl 

w 

[*  "RRJ 

- w 

["  -RR-J 

♦ sup 
|t-s|.6. 


EW 

* R^ 

- EW 

r ’RRJ 

2a,^(dn). 


(2.2) 


Hence,  (6^)  s ■ o(b(n))  for  each  k from  condition  (ii).  For 


c > 0 let 
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where  I. 


However, 


Since 


for  all 


where  | | 


A = [ sup 
" 2a^sS2b 


> e] 


1 

[ti_i,  tj]  with  = 2a  + for  i = 1,  ....  n’^.  Hence, 


(2.3) 


A c [ max 
n r 

ISiSn 


wear  ^k-=i\(b^)  I 


+ max  sup 

l^isn*^  sel. 

1 


HRHT  bferj"  \ bl^  ^ 


e] 


(2.4) 


max  sup 

isi^n’'^  sel, 
1 


HbW^k=l^”k(bfHr)  ■ ‘^k[b^)|  ^ RKT  W 


(2.5) 


(6jj)  = o(b{n))  by  condition  (ii),  there  exists  N(r)  such  that 

^k=l\(b(n))|  ^ 2^ 

n i N(r).  Using  the  basic  properties  of  sub-Gaussian  random  variables 
: k ■ 1,  2,  ...}  for  each  i] , for  each  n s N(r) 

t. 


P(A  ) s P[  max 


sbW  Ci\(bT^]  h 


^ ^i-l'’^  HJeW  ^k^l^fsT^)  h ^ 

i n'2  exp[-e^/4| [wj |„B  ] 


•11.  ■ »“P  |w(s)|  and 


n 

1 >2 


(2.6) 


[nb(n) 


nb  (n) 


To  obtain  the 
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complete  convergence  in  (2.1),  consider 


* ^n=N(r)+l^^^n^ 


^ * CN(r)+l 


-E^nb^ (n) 

■4imL 


^ N(r)  4 2n’'exp(-cn^) 


(2.7) 


2 

where  c ■ e /4||w||^.  Thus,  the  series  in  (2.7)  converges  by  the  integral  test. 


Lenma  2.  If  the  underlying  density,  f,  is  continuous,  then 


sup 

_e»<s<oa 


bfn) 


EW 


s - X, 


I b(n)  J 


- f(s) 


->■  0 as  n 00, 


Proof:  Since  f is  continuous  and  lim  f(x)  » 0 » lim  f(x),  given  e > 0 

X4«  X-4-00 

there  exists  6 > 0 such  that  lf(x)  - f(x')l  < e whenever  |x  - x’ | <6.  Let 
N be  sufficiently  large  so  that  |b(n)y|  < 6 for  all  n ^ N and  y e (a,  b] . 
Since  W(y)  » 0 for  y ^ [a,  b], 

fs  - X, 


bWH-RHTJ 


- f(s) 


(2.8) 


,W(y)f(s  - b(n)y)dy  - f(s)  ■ /W(y)[f(s  - b(n)y)  -f(s)]dy 


!!• 


< e />f(y)dy  ■ e 
a 


(2.9) 


unifomly  in  s for  all  n i N.  Hence, 


EW 


s - Xj 

■W1 


- f(s)  4.  0 


as  n 00, 
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Thus;  the  proof  of  Theorem  1 is  inmediate  from  Lemnas  1 and  2 since  for 
each  e > 0 

rs  - X, 


P[  sup 

-aa<s<oa 


^ w 

nb (n)  ^k=l 


bCn)  J 


- f(s) 


> e] 


S P[  sup 

_oo<s<cs 


s - X, 

s - X.l] 

1 rn 

W 

, 

k 

- EW 

1 

nb(n)  ^k=l 

b(n) 

b(n)  jj 

♦ P[  sup 


b(!n) 


EW 


s - X, 


bCn) 


f(s) 


C2.10) 


and  each  of  the  terms  in  (2.10)  is  a convergent  series  in  n.  All  of  the 
conditions  will  be  stated  in  Theorem  1 for  easy  reference. 


Theorem  1;  Let  {X^}  be  independent  random  variables  with  the  same  density 
function  fCs)  which  is  continuous  and  has  compact  support.  Let  W(x)  be  a 
nonnegative  weight  function  which  is  continuous  on  its  compact  support  and 
integrates  to  1.  If 


(a) 

(b) 


nb  (n)  > n 


"W 


2b  - 2a 


n’^b(n)J 


for  some  6 > 0,  and 

« o(b(n))  for  some  integer  r > 0,  then 


sup 

_«a<S<oa 


1 

nb(n) 


s - X 

f \ 

bdn) 


- f(s) 


-*■  0 


ccrplotcly  fis  n -*•  *. 

The  case  of  the  density  function  being  discontinuous  off  its  support  is 
not  entirely  excluded  in  Theorem  1.  The  following  steps  indicate  modifications 
which  allows  the  theory  to  include  a large  class  of  density  functions  [for 
example,  the  uniform  densities] . 
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Step  1.  For  an  unknown  density  function  which  is  continuous  on  [a,  b]  and 
vanishes  outside  [a,  b] , there  is  no  change  in  Lemma  1. 


Step  2.  In  Lamna  2 it  is  easy  to  verify  that  for  each  e > 0 

fs  - X,' 


sup 

a+eSs^b-e 


1 


b(n) 


W 


b(n) 


- f(s) 


as  n 


Step  3.  Combining  steps  1 and  2,  for  each  e > 0 

fs  - X, 


sup 

a+eSsSb-e 


1 y*'  IV 

nb(n) 


b(n)  J 


- f(s) 


conpletcly  as  n 

Mencc,  the  conplctc  convergence  of  the  maximal  deviation  of  the  density  estimate 
holds  on  arbitrary  closed  intervals  inside  of  [a,  b] . Similar  consideration 
was  also  given  in  Woodroofe  (1967) . 

In  Lemma  1 the  modulus  of  continuity  was  used  only  to  replace  fj^(s) 
by  a polygonal  approximation.  Thus,  the  following  corollary  can  be  obtained 
with  basically  the  same  proof. 


Corollary  1:  Let  the  density  function  f(s)  be  as  stated  in  Theorem  1. 
Let  W(x)  be  a nonnegative  weight  function  which  has  compact  support  and 
integrates  to  1.  If 


(a) 


nb^(n)  > n^ 


for  some  6 > 0, 


and 


(b*)  sup  lW(s)  - U i.(W)(s)|  = o(b(n)),  then 

asssb 


sup 

_co<s<°B 


- f(s) 


0 


co^nlctcly  as  n ■♦  *. 
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2 6 ' ' 

The 'Condition  nb  (n)  > n need  not  hold  for  all  n but  only  eventually. 

Also,  the  condition  can  be  stated  as 

OP 

(a')  / x’^exp(-cxb^(x))dx  < “> 

d 

for  some  d > 0 where  b(x)  is  a function  which  generates  the  bandwidths 
b(l),  b(2),  ...  and  c is  a constant. 

In  considering  mode  estimates,  assume  thai  the  continuous  density  function 
f(s)  has  a unique  mode  9,  that  is, 

f(0)  = max  f(s). 

-co<s«x> 

The  sample  mode  0^  is  also  assumed  to  uniquely  satisfy 

f (0^)  = max  f (s)  for  each  n. 

Theorem  2.  If  the  regularity  conditions  of  Theorem  1 or  Corollary  1 
(or  condition  (b'))  are  satisfied,  then 

|0^  - e|  ^ 0 

completely  as  n ^ 

Proof:  Since  f(s)  is  uniformly  continuous  and  has  a unique  mode  0,  for 
€ > 0 there  exists  n > 0 such  that  |x  - 0(  2 e implies  that  |f(0)  - f(x)|  2 q. 

Thus,  it  suffices  to  show  that  f(0jj)  f(9)  almost  surely.  But, 

|f(e^)  - fC9)|  s |f(ej  - f„(e„)|  * |f„(0J  - f(0)| 

s sup  lf(s)  - f (s) I + I max  f (s)  - max  f(s)| 

^ -<»<s<<»  ^ _ai><s<co 

s 2 sup  |f  (s)  - f(s)|  (2.11) 

-<B<S«a> 

pointwise  for  each  n.  From  (2.11)  and  the  cormlete  convergence  of  ||f  - f | | , 

it  follows  that  |0^  - 0 j -*•  0 completely. 
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3.  Comparisons  and  Useful  Weight  Functions.  Brief  comments  on  Nadaraya's 
(1965)  and  Itoodroofe's  (1967)  conditions  and  on  useful  weight  functions  which 
satisfy  the  results  of  the  paper  are  listed  for  comparison. 

To  obtain  a strong  law  rather  than  uniform  consistency  in  probability  (as 
listed  in  the  Introduction),  the  conditions  on  the  weight  function  and  bandwidth 
sequence  are  expected  to  be  more  stringent.  For  example, 

(A)  let  f be  an  unknown  continuous  density  function.  If 

OQ 

(Nl)  W(x)  = V/(-x)  is  of  bounded  variation  and  / x w(x)dx  exists;  and 

ZA  2 

exp  (-mb  (n))  exists  for  each  r > 0,  then 
||fj^  - f I 1^  -»■  0 with  probability  one  [Nadaraya  (1965)]. 

Next, 


(B)  let  the  density  have  support  on  some  neighborhood  of  compact  interval, 
e.g.,  [-1,  1].  If 
(Wl)  W e LIP(6),  0 < 6 S 1;  and 

(W2)  b(n)  = o(n)  and  n = o(b(n)"^)  with  1 < r < 6,  then 

' '’■-"kl' 


sup 

-15x^1 


nb(n) 


yn 

^k=l 


W 


X, 


b(!n) 


- f(x) 


= o(b(n)  ) with  probability  one 


where 


sup  / |y|w(x,  y)dy  -»■  0 as  t -*•  « 

-“xx<®  |yl^ 


[Woodroofe  (1967)]. 

For  the  results  of  this  paper,  the  weight  function  W(x)  needed  to  be 

continuous  on  its  compact  support  and  satisfy  a smoothness  condition  [(b)  or 

(b')].  Some  useful  weight  functions  for  these  results  will  now  be  listed. 

First,  Epanechnikov ' s (1969)  optimal  weight  function 

3 x^ 


W(x) 


4/5 


if  X s 5 


otherwise 
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can  be  used.  In  this  case,  let  a « -5  and  b « S.  Then,  lw(x)  - W(y) | s C|x  - y| 

^ ^*b^n^  constants  C and  C.  Thus,  condition  (b)  is 
easily  satisfied. 

If  the  weight  function  V/  satisfies  a Lipschitz  condition  of  order  a,  then 
|W(x)  - W(y)|  s M|x  - y|“  and 


“W 


f 1 1 . ..f  1 r 

b(nin^,  ^ ,b(n!)n^_ 


for  some  M > 0.  Thus,  the  bandwidth  sequence  b(n)  must  be  chosen  so  that 


.o-*-!-  . or 
b (n)n 


00  as  n • for  some  integer  r > 0. 


(3.1) 


Case  I;  -1  s a s 0.  ito  bandwidth  exists  for  (3.1). 

Case  2:  a > 0.  If  b(n)  ■ n"^  for  some  p > 0,  then  r is  an  integer 
i 2p(l  o)/a.  Then,  b^*°(n)n°’^  2 -►  »,  and  (3.1)  is  satisfied. 

Case  3;  a < -1.  Again,  if  b(n)  « n”^  for  some  p > 0,  then  r is  an  integer 
i p(a  ♦ l)/2a.  Then  b^*“(n)n°'*^  i ^ (3.1)  is  satisfied. 
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to  zero  is  obtained  under  varying  conditions  on  the  bandwidths  b(n)  and  smoothness  of  W.  For 
example,  one  choice  of  the  weight  function  is  Enanechnikov's  optimal  function  and  nb2(n) 

> n*  for  some  6 > 0.  The  uniform  cbnplete  convergence  of  the  node  estimate  is  also  considered.  •! 
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